Assessing the Use of Agent-Based Models for Tobacco Regulation by Robert Wallace

Author:Robert Wallace
Language: eng
Format: epub
Publisher: The National Academies Press
Published: 2015-07-15T00:00:00+00:00

Calibration, Verification, and Validation

To ensure that a model is valid and that it accurately represents the real world for its intended use, a model must go through calibration,30 verification,31 and validation processes at various points in model development. At this point in time, SnapDragon is very general and flexible. What the team described as verification entails a comparative analysis with empirical research and sensitivity analysis to determine what drives the model’s behavior (SNL, 2014a).32 These exercises are limited to internal validation or calibration of model parameters to replicate real-world results—what Berk (2008, p. 291) calls “internal quantitative credibility,” rather than confirmation that the model’s data-generating process is the same as the real-world data-generating process (Windrum et al., 2007). The SnapDragon modeling team members report that they have plans to conduct “parameter analysis and uncertainty quantification to make sure the parameters are consistent with knowledge of the system.”33 However, it is not clear whether even these exercises would constitute Berk’s external quantitative credibility—that is, comparisons of model output with real-world test data not used to develop and calibrate the model. The evaluations conducted so far have been “nearly data-free” (Berk, 2008, p. 293) relations between model output and ground truth that use very few stylized facts. Furthermore, these stylized facts are only qualitative rather than, say, quantitative values that draw from actual trends in smoking initiation and cessation over time. Missing so far is the search for areas where the model does well and where it fails in making predictions and then using these analyses to refine and improve the model.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.